How We Help You Ensure Data Accuracy

Accurate application security data is critical, and our goal is to continuously exceed industry standards in helping you make sure your data is accurate. That's why we've implemented various processes, procedures, and tools at every step, from designing and developing the plugins to importing and validating data. The following sections provide more information about how we help you ensure your data is accurate.

See also: How Permission Assist Helps Ensure Data Accuracy

Plugin Recon

Ensuring data accuracy begins with the development of a new plugin. When a client sponsors the development of a new plugin, we schedule a Recon session with the client and our Third-Party Integration Manager. The Recon session is a thorough discovery process in which we gather detailed information about all aspects of the application's security model. Within this process, we'll gather sample data, application documentation, and any other artifacts that can help us ensure data accuracy during the development process.

With a comprehensive understanding of the security data model, we can identify the most effective import strategy. Our primary concerns when determining an import strategy are as follows:

  • the completeness and accuracy of the data

  • the chain of custody - making sure, as much as possible, that the data comes directly from the vendor or the application

  • the consistency and availability of the data for all clients; the process of exporting the data must be repeatable and consistent (we will not use custom reports)

  • the format of the data

If the application's vendor has an approved way to export data natively, we'll use that method first. This keeps the chain of custody as clean as possible so you can provide proof for audit, compliance, and regulatory requirements. From there, we'll use the best technical solution based on what the application allows, what the vendor/application agreements allow, and what is best for all clients. Possible solutions include APIs, database connections, reports, and more.

If we determine that generated reports are the ideal way to import information, the following formats are supported (listed in order of preference, assuming no data is lost in a particular format):

  • Excel (XLSX OR XLS)

  • CSV

  • TXT

  • PDF (must be searchable; not rasterized)

  • HTML or XML

 

Plugin Development

During the development process, our Software Developers use a standardized set of processes and verifications to ensure the application's security data is accurate and working as expected. Some of the checks and verifications they use are listed below.

Enforcing Truths (Verifying data)

For each plugin, a series of tests is designed based on certain truths (things we know about the data with very high certainty). These tests are created within the plugin and used to ensure that the correct data is imported. Each application is different, but some examples of truths the plugin might test for include the following:

  • Report names or numbers - ensuring the correct report names, titles, or numbers appear in the proper format and location

  • Reviewing report headings - ensuring headings are in the proper format and location

  • Column names and headings - verifying that the required columns exist

  • Format and properties of the data - verifying that the data is in an expected format

If the data doesn't align with the tests during an import, the import will fail. To minimize import failures, we take a balanced approach to enforcing truths. This allows us to help ensure integrity while also allowing some flexibility as minor variations exist across clients or from one release to another.

Verifying Assumptions

Assumptions are things we know about the data, but with less certainty than a truth. By verifying assumptions, we make sure that what we think we know about the data is actually true. For example, let's say that during a recon session we learn that an application uses a role-based security model, and that it doesn't allow permission exceptions or overrides at the individual user level. We might consider that knowledge an assumption if we find evidence to suggest the assumption might not be true.

Each plugin is created with built-in verifications (sometimes called landmines) to continuously test assumptions. During an import, if the verification process detects something in the data that suggests an assumption isn't true, the import will fail. For example, using the same scenario described in the previous paragraph, the plugin could include verifications to confirm that permission exceptions or overrides do not exist at the individual user level. If the plugin detects exceptions or overrides at the individual user level, the import will fail.

Verifying assumptions is important for ensuring integrity. There have been times when a verification identified an issue with the data that needed to be reported to the vendor. There have also been times when an application update provided new data or changed the way the data was displayed, and a plugin update was needed. Regardless of the issue, we've got your back; we'll work with you to resolve the issue as quickly as possible.

Handling Unknowns

Unknowns are situations where we have enough evidence to realize we don't know everything yet. To handle unknowns, our developers create tests (similar to landmines). These tests are used to alert us if new data appears. For example, there have been times we've seen reports that could be helpful, but the report is blank, or a portion of the report is blank. We've also seen situations where modules, settings, or codes exist for future use. In these cases, we'll work with clients, vendors, and other experts to ensure these aren't errors in the data. After confirming the unknowns aren't errors, we can handle the unknowns by allowing the existing data to be imported as expected while also creating tests to look for the data we might see in the future. When new data does appear in an import, the import fails, and we can begin the Recon process. Each unknown is handled differently based on the situation, but the goal in all situations is to ensure that complete and accurate data is imported.

 

Quality Assurance

In addition to the checks and verifications that are built into the plugin, our development team goes through additional tests prior to releasing the plugin:

  • Automated unit tests - these tests are used to help ensure that all the permission expectations are completed correctly.

  • Integration tests - when all automated tests are passed, the developer completes a series of manual tests before adding the plugin to Sycorr's internal test environment. These tests evaluate the plugin as a whole to ensure everything is working as expected.

When all tests are passed, the developer submits the plugin and tests to our secure source code control system. With every source code change, all application plugins are automatically rebuilt, tested, and re-versioned for continuous integrity and stability using a continuous integration server. The versioning pattern is an industry standard of [major.minor.build].

 

Plugin Updates and Enhancements

We value continuous improvement and innovation. We understand that business needs and application functionality evolve, and we are committed to working closely with our clients, vendors, and other experts to assess and implement meaningful changes that enhance the client experience. The following paragraphs describe how each type of event is handled.

Updates

As applications evolve, plugins are updated to correct issues and to ensure that permissions are reported accurately. Some examples of when a plugin might be updated include the following:

  • An import fails with a defined error message

  • A client reports missing users, groups, or permissions

  • A client reports that the results are incorrectly formatted

When an update is requested, the Third-Party Integration Manager assesses the situation to determine next steps. For example, if an import fails due to a minor variation in the format of a report, the next step would involve sending a request to the development team. Whereas, if a client reports missing information, additional information is needed to ensure the accuracy and validity of the changes before sending a request to the development team. Regardless, all changes are handled in accordance with our best practices as described in the Plugin Recon, Plugin Development, and Quality Assurance sections.

 

Enhancements

Plugins are enhanced to add new features, improve existing features, or ensure that permissions are reported accurately. Some examples of when a plugin enhancement might be done include the following:

  • Additional support for optional file formats

  • Support for new optional files (adding context)

  • Addition of metadata

When an enhancement is requested, a Recon session is scheduled to gather more information. Additional research may need to be completed to gather information about the client's request, the impact of the changes, technological viability, and potential business impacts. All enhancements are handled in accordance with our best practices as described in the Plugin Recon, Plugin Development, and Quality Assurance sections.